Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add filesystem abstraction #251

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

Conversation

nenb
Copy link
Contributor

@nenb nenb commented Dec 13, 2023

WIP

The REST API has been updated to allow reading directly from fsspec-supported filesystems. The
new workflow is to first generate metadata on the server via a GET to the '/document' endpoint.
If the metadata references a document on a fsspec-supported path, then we go straight to the
'chats' endpoint. Otherwise, the workflow is as before and we must upload the document to the
server via a POST to the '/document' endpoint.
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@pmeier pmeier self-requested a review December 15, 2023 09:57
@pmeier
Copy link
Member

pmeier commented Dec 18, 2023

@nenb Just checking in to say that I didn't forget about this. I'll review as soon as I find the time.

@nenb
Copy link
Contributor Author

nenb commented Dec 18, 2023

@pmeier No problem. And just a reminder that this isn't really a PR, just a PoC (you asked me to open as a PR in #233 for ease-of-viewing), and so don't feel that you need to analyse too carefully. If there are any features that you think have potential, I am happy to pull these out as separate PRs.

@pmeier pmeier mentioned this pull request Dec 19, 2023
@pmeier
Copy link
Member

pmeier commented Dec 19, 2023

@nenb I've just written up #256, which I had in my backlog for two weeks now. Your proposal here fits nicely into that. Let's continue the discussion there.

@nenb nenb mentioned this pull request Aug 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants